Few-Shot Learning for Rooftop Detection in Satellite Imagery

Deep Learning Tutorial

Giorgio Coppola, Nadine Daum, Elena Dreyer, Nico Reichardt

Policy Relevance

  • Many public auhorities face the problem of limited labeled data
    • Annotation is expensive, slow, or requires domain expertise
  • Applications:
    • Medical sector: rare disease detection
    • Emergency management: flood extent mapping
    • Climate & energy: solar PV rooftop assessment
    • Urban planning: building footprints & infrastructure mapping
  • Few-shot learning (FSL) can help:
    • Learns to generalize from 1–5 labeled support examples per class
    • (In our case) learns feature embeddings and constructs class prototypes
    • Enables segmentation in a new city with minimal additional annotation

Problem Setting

  • Goal of the tutorial: apply Prototypical Networks to rooftop segmentation using only a few labeled tiles

  • Few-shot segmentation allows the model to learn characteristic rooftop shapes and textures from a small Geneva subset

  • Demonstrates how rooftop maps can be produced for solar potential estimation in a new geographic setting with limited labels

Dataset: Roofs of Geneva

  • Size: 1,050 labeled image-mask pairs

  • Task: Binary segmentation masks (rooftop vs background)

  • Geographic splits: 3 grids/ neighborhoods (North, Center, South)

  • Image size: 250x250 pixels

  • Categories: Industrial, Residential

Inside the dataset

Geneva Animation: raw image → overlay rooftop → binary mask

Few Shot Learning in General

Few-Shot Learning (FSL)

  • Learning new tasks, labels, or segmentations from very few labeled examples (N-way, K-shot)

Few-Shot Semantic Segmentation (FSSS)

  • Goal: Segment novel object classes using only a few annotated examples
  • Assigning a class label to every pixel

Prototypical Networks (ProtNets)

  • Learn a shared embedding space via a backbone model
  • Pixels belonging to the same class are close in feature space
  • Class representations are formed as prototypes
  • Training follows an episodic framework
  • Each episode consists of:
    • Support set: Few images with pixel-level masks Defines the target classes
    • Query image: Image where the model must segment the target classes

Prototypical Network Overview

Workflow

  • Support Image → Prototype → Similarity → Query Segmentation

Feature Extraction

  • Backbone: ResNet-18 CNN, pretrained on ImageNet
  • Projection: feature maps → embedding dimension (256 channels)

Evaluation Metric

\[ \mathrm{IoU} = \frac{|A \cap B|}{|A \cup B|} \]

Prototypical Network Overview

Modified figure from (Ding et al. 2022)

(Preliminary) Results

(1) Meta training loss

The “avg episode loss” at each epoch is the average cross-entropy error over all support–query tasks in that epoch. The encoder is successfully learning a feature space where prototype-based segmentation works increasingly well.

(Preliminary) Results

(2) Predicted masks

With 5-shot learning, the predicted masks have a mean IoU over 102 test samples of 0.485.

Here an example:

Discussion

Room for improvement:

  • Fine-tune / tweak model parameters
    • Add regularization
    • Increase number of epochs
  • Implement rough approximation of solar potential
    • e.g. based on IoU over roof area

Open for discussion:

  • Try a different encoder ?
    • e.g. ResNet-50
  • Change train / test split strategy ?
    • e.g. random shuffle regardless of geographic regions

GitHub Repo

References

  • Alsentzer, E., Li, M. M., Kobren, S. N., Noori, A., Undiagnosed Diseases Network, Kohane, I. S., & Zitnik, M. (2025). Few shot learning for phenotype-driven diagnosis of patients with rare genetic diseases. npj Digital Medicine, 8(1), 380. https://doi.org/10.1038/s41746-025-01749-1

  • Castello, R., Walch, A., Attias, R., Cadei, R., Jiang, S., & Scartezzini, J.-L. (2021). Quantification of the suitable rooftop area for solar panel installation from overhead imagery using convolutional neural networks. Journal of Physics: Conference Series, 2042(1), 012002. https://doi.org/10.1088/1742-6596/2042/1/012002

  • Chen, Y., Wei, C., Wang, D., Ji, C., & Li, B. (2022). Semi-supervised contrastive learning for few-shot segmentation of remote sensing images. Remote Sensing, 14(17), 4254. https://doi.org/10.3390/rs14174254

  • Ding, H., Zhang, H., & Jiang, X. (2022). Self-regularized prototypical network for few-shot semantic segmentation. Pattern Recognition, 132, 109018. https://doi.org/10.1016/j.patcog.2022.109018

  • Finn, C., Abbeel, P., & Levine, S. (2017). Model-agnostic meta-learning for fast adaptation of deep networks. In International Conference on Machine Learning (pp. 1126–1135). https://doi.org/10.48550/arXiv.1703.03400

  • Ge, Z., Fan, X., Zhang, J., & Jin, S. (2025). SegPPD-FS: Segmenting plant pests and diseases in the wild using few-shot learning. Plant Phenomics, 100121. https://doi.org/10.1016/j.plaphe.2025.100121

  • Hu, Y., Liu, C., Li, Z., Xu, J., Han, Z., & Guo, J. (2022). Few-shot building footprint shape classification with relation network. ISPRS International Journal of Geo-Information, 11(5), 311. https://doi.org/10.3390/ijgi11050311

  • Jadon, S. (2021). COVID-19 detection from scarce chest X-ray image data using few-shot deep learning. In Medical Imaging 2021 (pp. 161–170). https://doi.org/10.1117/12.2581496

  • Lee, G. Y., Dam, T., Ferdaus, M. M., Poenar, D. P., & Duong, V. (2025). Enhancing Few-Shot Classification of Benchmark and Disaster Imagery with ATTBHFA-Net. arXiv preprint arXiv:2510.18326. https://doi.org/10.48550/arXiv.2510.18326

  • Li, X., He, Z., Zhang, L., Guo, S., Hu, B., & Guo, K. (2025). CDCNet: Cross-domain few-shot learning with adaptive representation enhancement. Pattern Recognition, 162, 111382. https://doi.org/10.1016/j.patcog.2025.111382

  • Puthumanaillam, G., & Verma, U. (2023). Texture based prototypical network for few-shot semantic segmentation of forest cover: Generalizing for different geographical regions. Neurocomputing, 538, 126201. https://doi.org/10.1016/j.neucom.2023.03.062

  • Snell, J., Swersky, K., & Zemel, R. (2017). Prototypical networks for few-shot learning. Advances in Neural Information Processing Systems, 30. https://doi.org/10.48550/arXiv.1703.05175

  • Sung, F., Yang, Y., Zhang, L., Xiang, T., Torr, P. H., & Hospedales, T. M. (2018). Learning to compare: Relation network for few-shot learning. In CVPR (pp. 1199–1208). https://doi.org/10.1109/CVPR.2018.00131